Since the advent of Digital Video (DV) technology and high-quality DV camcorders there has been a continuing need for professional DV authoring tools. Various software packages, such as Apple Corporation's DVD STUDIO PRO™, include a suite of authoring tools and features that make Digital Video Disc (DVD) authoring relatively simple for video production companies, independent filmmakers, multimedia producers and the like. Users can shoot and edit video using a digital camcorder and an editing tool (e.g., Apple's Final Cut Pro™) to create the look and feel of menus, buttons and other embellishments. The video and audio are then organized into Titles, Chapters and Menus, encoded into formats required for DVD (i.e., MPEG-2, PCM, Dolby™ Digital Format) and burned to a DVD or written to other media that is sent to replication facilities.
DVD authoring tools typically run on high-end computer systems (e.g., Apple Power Mac G4) and provide an environment where DVD projects can be created and refined using built-in preview functions that enable the user to see how various links, menus, buttons and other embellishments integrate through the video in real time. Once the project is complete, the DVD can be burned to disc using an internal or external DVD-R drive or it can be sent to a service bureau or replication facility to write it to disc.
During project development, and particularly during the design of a DVD menu, an author will typically have various MPEG assets to represent chapters or one large MPEG asset for a whole title with embedded chapter markers. The author can choose a predefined template giving a layout or can define areas where chapter previews will play on a background. While experimenting with a layout (e.g., button positions, backgrounds, shapes for buttons, etc.), it is desirable to preview immediately after changes to gain an impression of how the playing DVD menu will look and feel on final disc. For inputs streams like DV or other QuickTime™ formats the ability to make and preview changes is a common process. If the source material, however, is already MPEG encoded (which is fairly typical in a DVD authoring environment), such functionality is more difficult to achieve because MPEG employs interframe compression techniques that do not permit most frames in a stream to be decoded independently of other frames. Moreover, MPEG streams vary in bit rate (e.g., 3-8 MBit/s, with variable bit rate being common), motion (which differs among scenes), picture structure (e.g., field encoded vs. frame encoded), resolution (e.g., MPEG1, cropped D1, half D1, 720×480/NTSC, 720×576/Pal, etc.) and frame rate (e.g., standard video frame rate vs. inverse telecined material).
The amount of processing power needed to decode each of the MPEG streams is also varying over time, with motion being the factor that influences processing power during different parts of the playback the most and VBR being second. Thus, decoding of multiple MPEG streams to provide synchronous playback of digital data provides additional challenges not addressed by conventional decoders.
For example, a DVD menu with six buttons typically requires the decoding of six video streams, i.e., one stream per button (assuming each button includes a video stream). If the background also presents a video stream, then the seventh stream needs decoding as well. When the menu screen is shown, it is expected that the motion in each of the six buttons will be played back in real time. Unfortunately, hardware limitations often dictate how many streams a given machine can decode completely in real time. Moreover, once the limit of processing is reached it is still desirable to preview the DVD menu, even if a preview is imperfect. As a compromise, some frames in MPEG streams are skipped causing the playback to stutter, but allows real-time playback of the MPEG clips to continue. A problem with skipping frames in multiple video streams is that if frames are skipped in some video streams but not others, the stream will fall out of sync with one another. One solution is to drop the same frames in all streams. Alternatively, frames that are close to each other in time between video streams can be dropped. However, in conventional systems, there is no mechanism for MPEG decoders working on various video streams to communicate with one another so that such frame-dropping can be coordinated. Regardless of how frames are dropped, when the interdependence of MPEG frames and other factors (e.g., bit rate, motion, resolution, frame rate, etc.) are taken into account the real time playback of an entire DVD menu with multiple streams is a complex process.
Accordingly, there is a need for a multi-stream decoder that can provide synchronous playback of digital data at a full or substantially full frame rate with low jitter, including digital data that has been compressed using interframe compression schemes (e.g., MPEG). The decoder should maximize the processing power of the host machine but still enable the user to perform editing tasks, such as selecting menus, moving windows, etc. In addition to digital video streams, at least one audio stream should be played back in sync with at least one video stream.